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1.0  EXECUTIVE  SUMMARY 


1.1  BACKGROUND 

Munitions  and  explosives  of  concern  (MEC)  contamination  is  a  high  priority  problem  for  the 
Department  of  Defense  (DoD).  Recent  DoD  estimates  of  MEC  contamination  across 
approximately  1,400  DoD  sites  indicate  that  10  million  acres  are  suspected  of  containing  MEC. 
Because  many  sites  are  large  in  size  (greater  than  10,000  acres),  the  investigation  and 
remediation  of  these  sites  could  cost  billions  of  dollars.  However,  on  many  of  these  sites  only  a 
small  percentage  of  the  site  may  in  fact  contain  MEC  contamination.  Therefore,  determining 
applicable  technologies  to  define  the  contaminated  areas  requiring  further  investigation  and 
munitions  response  actions  could  provide  significant  cost  savings.  Therefore,  the  Defense 
Science  Board  (DSB)  has  recommended  further  investigation  and  use  of  wide  area  assessment 
(WAA)  technologies  to  address  the  potential  these  technologies  offer  in  terms  of  determining  the 
actual  extent  of  MEC  contamination  on  DoD  sites  (DSB,  2003). 

This  report  describes  the  cost  and  performance  for  the  demonstration  of  two  WAA  technologies: 
synthetic  aperture  radar  (SAR)  and  hyperspectral  imaging  (HSI).  These  two  high-airborne  sensor 
technologies  were  demonstrated  at  Pueblo  Precision  Bombing  Range  (PBR)  #2  in  Otero  County, 
Colorado  (Eoley  et  ah,  2007a,  2007b).  Two  additional  high-airborne  sensor  technologies— light 
detection  and  ranging  (EiDAR)  and  orthophotography— were  also  demonstrated  at  Pueblo  PBR 
#2  (Foley,  2008),  and  the  use  of  SAR  and  HSI  as  WAA  technologies  were  evaluated  both  as 
standalone  sensor  technologies  as  well  as  members  of  a  WAA  sensor  suite. 

1.2  OBJECTIVES  OF  THE  DEMONSTRATION 

As  WAA  technologies,  SAR  and  HSI  are  not  designed  to  detect  individual  or  low-density  MEC 
contamination  but  rather  to  detect  and  map  former  high-density  regions,  such  as  bombing  targets. 
The  SAR  demonstration  was  conducted  to  determine  the  utility  of  the  dataset  to  detect  metal, 
such  as  munitions,  bomb  fragments,  tail  fins,  and  other  surface  metal  associated  with  prior 
military  training  activity.  The  HSI  demonstration  was  conducted  to  determine  the  utility  of  this 
dataset  for  SAR  false  alarm  mitigation,  detection  of  surface  metal  targets,  and  detection  of  target 
features.  Detection  of  surface  metal  objects  was  attempted  through  identification  of  spectral 
signatures  of  sufficient  spectral  contrast  with  background  signatures.  Detection  of  target  features 
and  other  large  munitions-related  features  was  evaluated  through  identification  of  secondary 
indicators  of  MEC  contamination  such  as  ground  disturbance,  vegetation  stress  and  composition, 
and  exposed  subsoil  patterns. 

1.3  REGULATORY  DRIVERS 

The  United  States  Army  Corps  of  Engineers  (US ACE)  is  the  lead  federal  agency  under  the 
Formerly  Used  Defense  Sites  (FUDS)  program.  USACE  administers  the  FUDS  Military 
Munitions  Response  Program  (MMRP)  using  DoD  investigation  and  cleanup  methods  based  on 
the  United  States  Environmental  Protection  Agency  (USEPA)  Comprehensive  Environmental, 
Response,  Compensation  and  Eiability  Act  (CERCLA)  process. 
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1.4  STAKEHOLDER/END  USER 


The  Environmental  Seeurity  Teehnology  Certifieation  Program  (ESTCP)  initiated  a  WAA  pilot 
program  in  2005  that  ineluded  the  former  Pueblo  PBR  #2  as  an  initial  demonstration  site.  This 
SAR  and  HSI  demonstration,  begun  in  2004  and  expanded  in  2005,  was  folded  into  the  pilot 
program.  As  part  of  this  program,  ESTCP  managed  the  stakeholder  issues.  ESTCP  used  a 
proeess  to  ensure  that  information  generated  by  the  high  airborne,  helieopter,  airborne,  and 
ground  validation  surveys  was  useful  to  a  broad  stakeholder  eommunity  (e.g.,  teehnieal  projeet 
managers;  federal,  state,  and  loeal  governments;  and  other  stakeholders). 

1.5  DEMONSTRATION  RESULTS 

A  primary  objeetive  of  this  demonstration  was  to  evaluate  SAR  as  a  tool  for  high-produetivity 
munitions  response  site  eharaeterization  (Eoley  et  ah,  2007a)  through  the  delineation  of  areas  of 
potential  MEC  eontamination.  Sky  Researeh  reengineered  existing  SAR  teehnology  into  a  new 
SAR  system  (denoted  as  Sky  SAR)  that  is  a  funetional  ultra-wideband  SAR  system  (230-400 
megahertz  [MHz])  eapable  of  eolleeting  aecurate  and  repeatable  data.  The  SkySAR  data 
eolleeted  for  this  demonstration  were  of  high  quality  with  moderately  low  noise  eharaeteristies, 
and  were  spatially  aeeurate,  predietable,  and  repeatable.  The  SAR  images  produeed  were  of  an 
output  pixel  size  of  0.5-m  provided  by  an  oversampling  faetor  approximately  30%  finer  than  the 
nominal  0.7-  by  0.7-m  resolution.  Viable  methods  of  data  fusion  were  developed  to  signifieantly 
reduee  vegetation  false  alarms  and  are  suitable  for  produetion-level  implementation. 

Multiple  passes  over  each  imaged  area  were  required  during  data  collection  to  ensure  detection 
of  objects  of  interest;  eight  passes  were  used  in  this  demonstration.  The  system  was  capable  of 
detecting  surface  targets  of  interest  of  sufficient  size,  including  simulated  ordnance,  emplaced 
targets,  and  in  situ  targets.  However,  the  size  of  items  that  can  be  reliably  detected  by  SAR  was 
found  to  be  limited  to  relatively  large  items  on  the  ground  surface.  Eor  the  Pueblo  demonstration 
site  areas  evaluated  in  this  demonstration,  metal  objects  about  30  centimeter(s)  (cm)  across  were 
found  to  be  detectable  in  the  SAR  data,  and  single  metal  objects  smaller  than  about  15  cm  in 
diameter  disappeared  in  the  clutter.  Therefore,  the  use  of  SAR  will  be  most  applicable  to  sites 
with  fairly  large  munitions  or  munitions  scrap  on  the  ground  surface.  In  addition,  SAR  needs  to 
be  used  in  conjunction  with  a  suite  of  WAA  sensors  for  target/clutter  discrimination  to  reduce 
the  false  alarm  rate  (PAR).  Pinally,  processing  and  analysis  of  SAR  data  sets  is  not  yet  an 
automated  process  and,  consequently,  can  be  time-consuming  and  labor-intensive. 

HSI  sensors  were  also  deployed  from  fixed-wing  platforms  as  part  of  the  high  airborne  data 
collection  and  analysis  for  WAA.  The  HSI  data  for  this  project  were  acquired  at  spatial 
resolutions  of  1.5  and  3  2/pixel.  HSI  data  collection  augmented  the  analysis  of  SAR  in  a 
multiple-sensor  fusion  process  to  reduce  the  PAR  through  the  generation  of  detailed  vegetation 
models  used  to  discriminate  metal  from  vegetation  in  SAR  surface  object  detections.  It  was 
found  that  the  1 .5  m2  spatial  resolution  data  was  needed  for  vegetation  modeling. 

In  addition,  the  use  of  HSI  imagery  for  detecting  and  extracting  large  munitions-related  features 
such  as  target  aiming  features  and  detection  of  landscape  disturbance  patterns,  including  surface 
disturbance  features  difficult  to  observe  in  other  datasets,  was  evaluated.  Although  the  HSI 
sensor  alone  would  not  outperform  the  EiDAR/orthophotography  sensor  combination  for  general 
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WAA  site  characterization,  the  HSI  imagery  was  found  to  be  useful  in  detecting  some  large 
features  obscured  in  other  remote  sensing  datasets.  Last,  HSI  imagery  was  also  evaluated  with 
respect  to  the  direct  detection  of  metallic  objects  such  as  munitions  and  munitions  scrap.  The  use 
of  hyperspectral  data  to  detect  surface  concentrations  of  metal  materials  depends  on  the  size  and 
spectral  characteristics  of  the  metal  objects  to  be  detected  as  well  as  the  spectral  characteristics 
of  the  background  against  which  they  must  be  detected.  At  the  demonstration  site,  background 
soils  with  abundant  iron  content  made  direct  detection  of  surface  metal  concentrations  using  HSI 
problematic.  The  final  report  for  the  HSI  demonstration  provides  a  quantitative  analysis  of  this 
phenomenological  issue  and  recommendations  regarding  the  spatial  and  spectral  resolution 
requirements  and  detection  phenomenology  parameters  necessary  to  make  direct  MEC  detection 
with  HSI  useful  in  a  WAA  scenario  (Foley  et  ah,  2007b). 
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2.0  TECHNOLOGY  DESCRIPTION 


2.1  TECHNOLOGY  DEVELOPMENT  AND  APPLICATION 

2.1.1  SAR 

The  SAR  technology  utilized  on  this  demonstration  project  by  Sky  Research  is  a  redeveloped 
and  refurbished  ultra-wideband,  ultrahigh-frequency  (UHF)  radar  system  based  on  the  DoD 
FOLPEN  III  system  developed  by  Stanford  Research  Institute  for  the  Defense  Advanced 
Research  Projects  Agency  (DARPA)  in  1993  (Kemer,  1999).  Sky  Research  obtained  this  system 
from  the  Naval  Surface  Warfare  Center,  Dahlgren  Division,  began  upgrades  in  September  2004, 
and  deployed  the  reengineered  SkySAR  in  May  2005.  The  number  of  system  redesigns  was 
limited  by  this  7-month  development  effort;  consequently,  SkySAR  was  developed  as  a  proof-of- 
principle  SAR  with  limited  automated  and  user  programmable  operating  functions. 

2.1.2  HSI 

The  first  airborne  image  spectrometer  was  deployed  by  the  National  Aeronautics  and  Space 
Administration  (NASA)  in  1985.  The  first  commercially  available  HSI  was  the  compact  airborne 
spectrographic  imager  (CASI)  in  1989.  Since  then,  HSI  has  been  used  in  environmental  mapping 
applications  to  define  surface  features  such  as  plant  species,  soil  types,  and  moisture  content 
(Ritter  et  ah,  1996)  by  imaging  and  analyzing  surface  features  over  a  wide  spectral  band  using 
advanced  charged  coupled  device  (CCD)  sensor  technology  and  imaging  spectrograph  optics. 

The  HyMap  HSI  technology  used  for  this  demonstration  has  been  in  commercial  and  scientific 
use  for  geological  resource  mapping,  environmental  monitoring,  agriculture,  and  various 
research  applications  since  1999.  In  2000,  Sky  Research  used  HSI  technology  to  discriminate 
vegetation  in  SAR  datasets  at  the  Former  Lowry  Bombing  and  Gunnery  Range  (FLBGR)  near 
Aurora,  Colorado;  however,  the  Pueblo  demonstration  is  the  first  attempted  use  of  an  HSI  sensor 
to  directly  detect  ferrous  surface  metal  for  MEC  site  characterization  and  the  first  successful  use 
for  detecting  munitions  target  features  not  visible  in  conventional  orthophotography  or  high- 
resolution  topographic  data. 

2.2  TECHNOLOGY  DESCRIPTION 
2,2.1  Aircraft  Platforms 

Eor  this  demonstration,  both  sensors  were  mounted  on  fixed-wing  plane  platforms.  The  SkySAR 
was  installed  in  an  equipment  pod  on  the  Sky  Research  Jetstream  31  aircraft  (Eigure  1).  The  HSI 
sensor  was  deployed  on  a  DHC-6/300  Twin  Otter  aircraft. 
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Figure  1,  The  Sky  Research  J31  Aircraft  Was  Used  for  All  Aviation  Activities  for  SAR 
Data  Collection,  (The  Equipment  Pod  [lower-left]  Houses  the  Bi-Static  Horns  Antenna  [right].) 

2,2,2  SAR  Technology 

Deployed  from  a  fixed-wing  platform,  SAR  technology  measures  round  trip  travel  time  and 
strength  of  pulsed  microwave  signals  emitted  by  a  radar  antenna  and  reflected  off  a  distant 
surface  or  object.  An  image  is  built  as  pulses  are  emitted,  returned,  and  recorded  along  the  flight 
line  of  the  aircraft.  SAR  data  can  be  processed  in  several  ways,  yielding  different  data  products. 
Single  polarization  data  allow  creation  of  2D  images.  Multiple  polarization  data  allows  more 
detailed  discrimination  of  different  surface  types  or  targets.  Interferometric  processing  allows 
creation  of  a  digital  elevation  model  (DEM). 

The  functional  capabilities  of  SAR  to  detect  surface  munitions  or  other  metal  targets  of  interest 
are  dependent  on  the  size  of  the  target  and  the  radar’s  angle  of  illumination  of  the  target. 
Atmospheric  conditions  such  as  clouds,  rain,  or  fog  do  not  significantly  reduce  SAR  capabilities. 
However,  the  degree  and  type  of  vegetation  on  the  site  and  topographic  setting  of  the  scene 
introduces  factors  that  can  attenuate,  scatter,  or  misposition  the  SAR  returns.  The  separation  of 
SAR  signals  originating  from  surface  and  subsurface  metal  from  those  generated  by  vegetation 
or  topographic  effects  represents  a  major  challenge  to  SAR  in  this  application. 


6 


2.2.3  HSI  Technology 


The  HSI  system  used  in  this  demonstration  was  a  HyMap  operated  by  HyVista  Corporation  of 
Sydney,  Australia.  While  a  number  of  airborne  imaging  spectrometers  exist  and  the  technology 
is  in  active  commercial  use  for  a  variety  of  geological  and  vegetation  mapping  efforts,  the 
HyMap  sensor  offered  the  best  available  combination  of  spectral  and  spatial  resolution,  signal- 
to-noise  ratio  (SNR),  and  geospatial  accuracy  necessary  to  achieve  the  demonstration  objectives. 
However,  it  should  be  noted  that  use  of  the  HSI  data  for  vegetation  modeling  for  SAR  false 
alarm  reduction  was  found  to  be  labor-intensive. 

The  HyMap  is  an  airborne  hyperspectral  sensor  that  measures  the  electromagnetic  spectrum  from 
0.45  micrometer  (pm)  to  2.5  pm  in  126  separate  but  contiguous  bands  that  have  variable  widths 
(from  15  nanometers  [nm]  in  the  visible  wavelengths  to  17  nm  in  the  short-wave  infrared 
[SWIR]  wavelengths).  HyMap  records  an  image  by  using  a  rotating  scan  mirror  that  allows  the 
image  to  build  line  by  line  as  the  aircraft  flies  forward.  The  reflected  sunlight  collected  by  the 
scan  mirror  is  then  dispersed  into  different  wavelengths  by  four  spectrometers  in  the  system.  The 
spectral  and  image  information  from  the  spectrometers  is  digitized  and  recorded  on  tape.  The 
many  contiguous  bands  of  a  hyperspectral  imager  produce  complete  spectral  signatures 
associated  with  each  recorded  pixel  (Figure  2)  that  allow  for  identification  of  materials  rather 
than  simple  discrimination  afforded  by  most  space-based  remote  sensing  instruments  (e.g., 
Landsat)  or  airborne  multispectral  instruments.  Figure  3  shows  the  reflectance  spectrum  for 
several  different  classes  of  materials  as  a  function  of  wavelength  over  the  bands  (spectral 
analysis  conducted  on  Pueblo  1.5-m  resolution  data). 


Figure  2,  The  General  Hyperspectral  Imaging  Concept  (measurement  of  reflected 
electromagnetic  radiation  across  many  tens  of  contiguous  bands  where  each  pixel  contains  one 
complete  spectral  signature  used  to  absolutely  identify  surface  materials). 
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Figure  3,  HyMap  Spectral  Signatures  (126  bands)  for  Three  Material  Groups  of 
Interest— Dry  Vegetation,  Green  Vegetation  and  Iron  Oxide  (important  absorption  features 
indicated,  and  atmospheric  absorption  regions  shown  in  blue). 

To  minimize  distortion  induced  in  the  image  by  aircraft  pitch,  roll,  and  yaw  motions,  the  HyMap 
is  mounted  in  a  gyro-stabilized  platform.  While  the  platform  minimizes  the  effects  of  aircraft 
motion,  small  image  distortions  remain.  These  residual  motions  are  monitored  with  an  inertial 
measurement  unit  (IMU).  Dedicated  geocorrection  processing  restores  the  full  geolocation 
information  and  allows  the  creation  of  geographic  information  system  (GlS)-ready  products. 

2.3  PREVIOUS  TESTING  OF  THE  TECHNOLOGIES 

DARPA  collected  SAR  data  at  a  site  where  emplaced  munitions  items  simulated  regions  of 
relatively  high  target  density.  Despite  the  fact  that  the  DARPA  SAR  system  was  designed  for 
sensing  tanks  in  foliage  rather  than  for  high-density  MEC,  the  munitions-sensing  results  were 
encouraging  (Carin,  2002).  Specifically,  models  developed  by  Duke  University  predicted 
signatures  that  were  in  close  agreement  with  what  DARPA  measured,  and  the  Duke 
segmentation  algorithms  delineated  high-density  MEC  regions  from  foliage. 

ESTCP  provided  funding  to  the  U.S.  Army  Engineer  Research  and  Development  Center  to 
determine  the  feasibility  of  using  the  foliage  penetration  (EOPEN)  SAR  system  to  measure  MEC 
target  signatures  in  various  settings  with  a  secondary  objective  of  determining  if  the  EOPEN 
SAR  has  applications  for  MEC  range  delineation  (Simms,  2003).  The  EOPEN  system  was  found 
to  be  capable  of  imaging  the  larger  targets;  groups  of  surface  155-mm  projectiles  were  observed 
but  individual  items  were  not.  The  bandwidth  of  the  UHE  portion  of  the  UHE-VHE  EOPEN  SAR 
at  200-500  MHz  was  similar  to  the  SkySAR  operating  range  of  230-440  Mhz,  and  the  radar  cross 
section  (RCS)  obtained  by  the  EOPEN  processing  methods  (designed  to  use  both  the  UHE  signal 
and  a  companion  VHP  SAR  signal  to  detect  vehicle-sized  objects  under  a  tree  canopy)  were 
suitable  for  detecting  only  clusters  of  155-mm-sized  objects.  Sky  Research  collected  SAR  data  at 
FLBGR  in  Colorado.  Unfortunately,  much  of  ELBGR  was  surface  cleared  prior  to  SAR 
collection,  and  therefore  this  dataset  did  not  allow  consideration  of  WAA  of  high-density  MEC 
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regions,  although  the  proeessing  results  did  provide  proof-of-eoneept  for  the  Bayesian  fusion 
arehiteeture. 

The  basie  HSI  data  eolleetion  teehnology  used  for  this  demonstration  has  been  well  established 
by  HyVista.  Prior  to  instrument  deployment  in  this  demonstration,  speeifications  and  sample 
datasets  for  this  individual  instrument  were  examined  and  eompared  with  HSI  data  eolleeted 
with  other  instruments  for  previous  Sky  Researeh  projeets. 

2,4  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGIES 

The  primary  advantage  of  using  an  airborne-based  system  in  general  is  the  ability  to  aequire  a 
large  amount  of  data  eovering  a  wide  area  in  a  relatively  short  period  of  time.  Data  eolleetion  for 
thousands  of  aeres  for  site  evaluation  takes  a  matter  of  days,  as  eompared  to  the  mueh  longer 
time  frame  for  eolleeting  ground-based  data.  SAR  was  found  to  deteet  surfaee  metal,  but  was 
limited  by  the  size  of  the  items  of  interest  that  were  detected  and  by  false  alarms. 

The  main  advantage  of  HSI  is  in  generating  highly  detailed  infrared  (IR)  +  visible  reflectance 
spectral  data  for  individual  geolocated  pixels  on  the  ground  surface.  Because  these  spectra  are 
highly  correlated  with  chemical  composition,  HSI  data  can  be  used  to  create  vegetation  models 
for  SAR  false  alarm  mitigation;  to  identify  soil  and  vegetation  disturbance  patterns  that  can  be 
caused  by  munitions  impacts;  and  possibly  to  identify  metallic  munitions  components  and 
chemical  salts  in  the  soil.  However,  HSI  technology  is  limited  by  spatial  and  spectral  resolution 
and  geocorrection  accuracy. 

Both  SAR  and  HSI  datasets  are  large  and  complex  and  thereby  cost-  and  labor-intensive  to 
process  and  analyze.  In  addition,  they  require  the  information  technology  infrastructure  to 
manage,  process,  and  store  the  data.  These  limitations  are  expected  to  diminish  as  new  or 
improved  technologies  are  developed  in  response  to  the  rapidly  growing  market  for  high- 
resolution  geospatial  data  in  a  broad  spectrum  of  applications  and  as  computing  resources 
continue  to  expand  in  speed  and  power. 
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3.0  DEMONSTRATION  DESIGN 


3.1  PERFORMANCE  OBJECTIVES 

Performance  objectives  are  critical  demonstration  components  because  they  provide  the  basis  for 
evaluating  the  technology  performance.  For  these  demonstrations,  performance  criteria  were 
established  and  documented.  The  SAR-related  performance  criteria  were  as  follows: 

•  Horizontal  accuracy  of  the  SAR  geocorrected  and  coregistered  imagery  relative  to 
established  ground  control 

•  System  noise  level  quantification 

•  SAR  response  characteristics  of  emplaced  simulated  munitions  targets 

•  SAR  response  characteristics  of  in  situ  items  (metal  and  vegetation)  surveyed 
within  the  calibration  areas. 

For  HSI,  the  primary  performance  objectives  relate  to  the  attainment  of  resolution  and  coverage 
specifications  for  the  basic  HSI  data  collection.  Secondary  performance  objectives  relate  to  the 
contribution  HSI  datasets  provide  to  the  WAA  process,  including: 

•  Mitigation  of  FAR  in  the  SAR  data  through  contribution  of  spectral  information 
to  a  multiple-sensor  vegetation  model 

•  Detection  of  bombing  target  features  and  other  large  munitions-related  features 

•  Detection  of  surface  concentrations  of  munitions-related  metallic  materials. 

3.2  TEST  SITE  SELECTION 

Pueblo  PBR  #2  was  initially  selected  for  the  MM-0416  demonstration  of  remote  sensing 
technologies  in  2004  (SAR,  HSI,  LiDAR,  and  orthophotography)  based  upon  site  characteristics 
that  were  expected  to  be  amenable  to  metal  and  feature  detection  in  high  airborne  remote  sensing 
datasets.  When  ESTCP  created  the  WAA  pilot  program  in  2005  in  response  to  the  DSB  Task 
Force  report  and  Congressional  interest,  Pueblo  PBR  #2  was  selected  as  one  of  the  first 
demonstration  sites.  Consequently,  the  demonstration  area  for  Pueblo  PBR  #2  was  expanded  and 
a  second  data  collection  conducted  in  2005.  This  second  data  collection  included  collection  of 
HSI,  LiDAR,  and  orthophotography  datasets  and  encompassed  a  second  documented  bombing 
target  and  a  suspected  bombing  target  within  the  demonstration  site  boundaries  (USAGE,  1995). 

3.3  TEST  SITE  HISTORY/CHARACTERISTICS 

Pueblo  PBR  #2,  located  in  the  southern  part  of  Otero  County,  Colorado,  was  used  as  a  World 
War  Il-era  military  training  facility.  Within  the  105  square-mile  (67,770  acres)  PUDS,  the  7,500- 
acre  WAA  demonstration  area  encompasses  the  two  bombing  targets  (BT3  and  BT4)  and  the 
suspected  75-mm  air-to-ground  target  area  documented  in  the  Archive  Search  Report  (ASR) 
(USAGE,  1995)  and  shown  in  Pigure  4.  Munitions  that  have  been  found  on  Pueblo  PBR  #2 
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Figure  4,  The  WAA  Demonstration  Area  (in  yellow)  Was  Located  Within  the  Former  Puehlo  PER  #2  in  Otero  County, 

Colorado, 


Munitions  Response  Area  (MRA)  and  were  doeumented  in  the  ASR  include  100-lb  general 
purpose  and  practice  bombs,  75-mm  shot,  and  50-caliber  small  arms  ammunition  (USAGE, 
1995).  The  boundaries  of  the  HSI  data  collection  and  the  SAR  imagery  output  areas  are  shown  in 
Figure  5. 

Land  within  the  study  area  is  primarily  in  federal  ownership  managed  by  the  U.S.  Forest  Service 
as  the  Comanche  National  Grasslands  with  portions  leased  to  private  owners  or  owned  by  the 
State  of  Colorado.  Somewhat  less  than  2,000  acres  of  the  study  area  are  privately  owned, 
nonresidential  grazing  lands. 

Although  initially  considered  to  be  a  benign  site  for  SAR  and  HSI  data  collection  due  to 
relatively  simple  physiography  and  sparse  vegetation,  the  site  provided  a  more  challenging 
demonstration  environment.  First,  vegetation  was  an  issue  with  respect  to  SAR  false  alarms.  Due 
to  the  relatively  dry  climate,  prairie  grass,  and  succulent  desert  vegetation  (including  yucca, 
cholla  and  prickly  pear  cactus,  juniper,  and  pinyon  pine  trees),  as  well  as  other  species  of  woody 
or  succulent  vegetation  are  present  at  the  site.  Plants  with  taproots,  such  as  yucca,  have  high 
water  content  that  can  generate  SAR  responses  and  cause  false  alarms.  Second,  surface  metal 
detections  using  HSI  were  not  possible.  Modern  spectral  processing  methods  can  achieve 
materials  detection  using  sub-pixel  techniques  for  materials  that  consist  of  as  little  as  10%  of  a 
pixel;  however,  this  requires  sufficient  spectral  contrast  between  the  target  material  and  the 
background.  At  the  demonstration  site,  the  abundant  iron  content  of  the  background  soils  did  not 
provide  enough  spectral  contrast  for  identifying  iron. 

3.4  PHYSICAL  SETUP  AND  OPERATION 

3.4.1  Mobilization/Demobilization 

Mobilization  and  demobilization  for  these  demonstrations  required  transport  of  the  plane, 
equipment,  pilot,  and  sensor  operators  to  and  from  the  base  of  operations.  Ground  personnel 
deployed  from  Denver,  Colorado,  to  establish  ground  fiducials,  establish  and  operate  global 
positioning  systems  (GPS)  base  stations,  and  provide  logistical  support  were  returned  to  Denver 
at  the  conclusion  of  the  demonstrations. 

3.4.2  Calibration  Areas 

Two  discrete  calibration  sites  were  established  in  the  vicinity  of  the  BT4  area.  The  In  Situ 
Calibration  Area  was  a  90x1 12-m  (2.46-acre)  area  with  known  occurrences  of  surface  munitions 
debris.  Prior  to  data  acquisition  surveys,  coordinate  data  was  collected  in  this  area  for  all 
vegetation,  rock,  surface  metal,  and  microtopographic  features.  The  Simulated  Ordnance 
Calibration  Area  was  a  108xl37-m  (3.67-acre)  area  with  typical  vegetation  and  terrain  without 
previously  known  munitions  debris.  Twenty-eight  metallic  items  that  varied  in  size,  shape  and 
material  to  simulate  intact  munitions  objects  (sizes  ranging  from  approximately  13 -in  x  3 -in  to 
60-in  X  18-in)  were  emplaced  on  the  ground  surface  in  this  area  as  calibration  items. 
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Figure  5,  HSI  Survey  Boundaries  and  SAR  Image  Output  Areas  at  Pueblo  PBR  #2, 
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3.4,3  Ground  Control/Positioning 
3.4,3.1  SAR 


Triangular-plate  trihedral  eomer  refleetors  and  top  hat  fiducial  targets  were  used  for  ground 
control  at  the  site.  Geographic  coordinates  of  these  locations  were  obtained  by  a  licensed 
surveyor  using  real-time  kinematic  GPS  (RTK  GPS).  Two  RTK  GPS  base  stations  were  operated 
during  the  data  collection  survey.  Onboard  positioning  was  achieved  with  an  integrated  position 
and  orientation  (POS)-AV  system  with  post-processed  dual  channel  RTK  GPS. 

3.4,3.2  HSI 


No  separate  ground  control  was  used  during  HyMap  surveys.  All  spatial  correction  information 
was  collected  on  board  the  aircraft  using  a  C-MIGITS  II  IMU,  and  geocorrections  were 
accomplished  post-survey  using  the  GPS  information  collected  during  the  mission.  An 
assessment  of  geolocation  accuracy  indicates  that  the  basic  accuracy  achieved  was  on  the  order 
of  1-1.5  pixels.  Additional  geocorrection  using  control  tie  points  from  the  orthophotography  was 
completed  to  increase  the  spatial  accuracy  of  the  HSI  data  for  spectral  sharpening  of  the  high- 
resolution  orthophoto  and  LiDAR  datasets,  as  well  as  for  the  vegetation  modeling  conducted  for 
SAR  FAR  mitigation. 

3.4.4  Navigation  Systems 

An  Applanix  AA^  POS  system  was  co-mounted  with  the  SkySAR  to  record  the  aircraft’s  GPS 
position  using  a  dual  frequency  GPS  receiver  and  attitude  (pitch,  roll,  and  yaw)  applying  an 
IMU.  A  customized  pilot  guidance  system  was  used  for  flight  navigation. 

3.4.5  Period  of  Operation 

Test  flights  of  the  SAR  system  were  conducted  in  April  of  2005.  SAR  data  acquisition  flights 
were  conducted  between  April  30  and  May  6,  2005,  as  conditions  allowed  during  intermittent 
periods  of  clear  weather.  Data  processing  and  analysis  were  conducted  in  2005  and  2006. 

HSI  data  was  collected  on  September  17,  2004,  and  August  30,  2005.  Data  processing  and 
analysis  of  the  2004  HSI  data  were  conducted  in  2004  and  2005.  Data  processing  and  analysis  of 
the  2005  data  were  conducted  in  2005  and  2006. 

3.4.6  Operating  Parameters  for  the  Technology 

The  SkySAR  data  collection  parameters  and  sensor  parameters  are  provided  in  Table  1  and  the 
HSI  data  collection  parameters  in  Table  2. 
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Table  1.  SkySAR  Data  Collection  and  Sensor  Parameters, 


Parameter 

Value 

Units  /  Notes 

Flight  speed 

150 

knots 

Flight  altitude 

1,000 

m  above  ground  level  (AGL) 

Swath  width 

1 

kilometers  (km) 

Frequency 

230-440 

MHz 

Transmitted  power  (peak) 

250 

watts  (W) 

System  noise  figure 

5.0 

decibels  (dB) 

Look  angle 

30-60 

degrees 

Ground  pixel  size 

0.7  X  0.7 

m 

Operation  duration 

11 

hours 

Total  area 

27,455 

acres 

Number  of  look  directions 

8 

NS,  NS,  EW,  WE,  NE,  NE,  SW,  NW 

Effective  coverage 

3,432 

acres/hour 

The  total  area  is  the  number  of  acres  surveyed  in  each  of  the  8  look  directions.  The  demonstration  area  was  approximately  2,225  acres. 


Table  2.  HSI  Data  Collection  Parameters, 


Parameter 

Value 

Units  /  Notes 

Flight  speed  (1.5-m  resolution) 

82 

knots 

Flight  altitude  (1.5-m  resolution) 

600 

mAGL 

Flight  speed  (3-m  resolution) 

100 

knots 

Flight  altitude  (3-m  resolution) 

1,200 

mAGL 

Swath  width 

1 

km 

Instantaneous  field  of  view  (IFOV) 

2.5 

millirad  (mrad) 

Field  of  view  (FOV) 

61.3 

degrees 

Wavelengths 

0.45-2.5 

fim 

SNR 

>1,000:1 

Total  area  -  2004  data  collection 

6,710 

acres 

Total  area  -  2005  data  collection 

9,628 

acres  (includes  an  overlap  of  3,004  acres  of  data 
collected  in  2004) 

3.5  ANALYTICAL  PROCEDURES 
3,5.1  SAR  Data  Processing  and  Analysis 

POS  and  the  RTK  GPS  data  are  parsed  to  ereate  input  veetors  for  image  formation.  Image 
formation  ineludes  range  eompression,  motion  eompensation,  and  azimuth  eompensation  steps 
(Foley,  2007a).  Seeondary  proeessing  aetivities  ineluded  eonverting  the  georeetified  SAR 
imagery  from  binary  eomplex  data  files  to  georefereneed  image  files,  assessing  spatial  aeeuraey 
relative  to  ground  referenee  fidueials,  performing  any  required  eoregistration,  and  integrating  the 
imagery  into  the  projeet  GIS  framework.  Figure  6  provides  a  fiowehart  overview  of  the  image 
formation  proeess.  Appendix  A  shows  the  SAR  imagery  for  the  entire  Pueblo  demonstration 
area. 

Spatial  aeeuraey  was  assessed  by  determining  spatial  offsets  between  preeision  top  hat  fidueial 
loeations  and  the  peak  amplitude  loeations  of  the  eorresponding  feature  in  eaeh  SAR  image. 
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Targets  were  selected  in  a  modeling  process  by  combining  eight  coregistered  cardinal  pass 
direction  images  into  a  single  image  using  a  maximum  operation,  where  targets  were  extracted 
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Figure  6,  SkySAR  Data  Processing  Flow  Schematic, 


based  on  an  amplitude  threshold  operation  on  the  eombined  image.  Target  loeations  were  then 
eomputed  based  on  the  eentroid  of  the  thresholded  amplitude  region,  and  target  morphology 
attributes  were  also  eomputed  from  the  thresholded  “sliee”  region  in  the  modeling  environment. 

To  maximize  the  potential  to  utilize  SAR  in  WAA,  differentiation  between  suspeeted  MEC 
(metal)  and  naturally  oeeurring  elutter  (frequently  vegetation)  was  needed  beeause  both  metal 
and  elutter  have  the  potential  to  exhibit  high  energy  SAR  responses.  Figure  7  provides  a 
sehematie  diagram  of  this  proeess.  The  two-stage  elassifieation  system  included  a  front-end  pre- 
screener  to  support  moderate-to-high  surface  MEC  detection  rates  while  providing  good  false 
alarm  mitigation.  The  prescreener  compared  the  SAR  response  from  each  of  the  simulated 
ordnance  items  to  background  response  in  the  vicinity,  serving  to  identify  points  of  interest  (POI) 
for  further  processing  by  a  multisensor,  multifeature  classifier  designed  to  significantly  suppress 
false-alarm  rates  to  support  WAA  requirements. 

3.5,2  HSI  Data  Processing  and  Analysis 

A  variety  of  software  packages  were  used  for  processing  the  HSI  data  (Foley,  2007a).  The 
processed  HSI  data  were  used  for  vegetation  modeling,  detection  of  surface  metal,  and  detection 
of  munitions-related  features.  To  analyze  the  HSI  data,  ENVI  Minimum  Noise  Fraction  (MNF) 
transformations  (principal  components-based  algorithm)  were  run  on  all  1.5-m  resolution 
flightstrips,  converting  the  126-band  reflectance  data  to  reduced  band  sets  of  20  eigenbands. 
These  MNF  images  were  then  geocorrected  using  the  geographic  lookup  table  (GET),  and 
converted  to  band  sequential  (BSQ)  raster  format  for  use  in  the  ArcGIS  visualization  and  feature 
extraction  environment.  Band  combinations  were  mapped  to  the  red-green-blue  (RGB)  image 
rendering  channels  after  determining  which  bands  and  band  combinations  provided  the  best 
feature  visualization  for  detecting  known  target  features.  The  entire  site  was  then  systematically 
reviewed  using  single  bands  and  band  combinations,  in  conjunction  with  other  imagery  derived 
from  FiDAR  and  orthophotography  datasets. 


Veg  Height 

_ 

SAR  data 

Prescreener 

Points  of  Interest 

_ 

Orthographic 

Possible  UXO 
locations 


Figure  7.  Two-Stage  Classification  System  Included  a  Front-End  Prescreener  to  Support 
Moderate-to-High  MEC  Detection  Rates  While  Providing  Good  False  Alarm  Mitigation, 
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3.5,3  Vegetation  Modeling 


Vegetation  modeling  was  condueted  using  HSI  data  in  eonjunction  with  LiDAR  and 
orthophotography  data  for  the  SAR  analysis  to  generate  feature  deteetions  for  speeific  vegetation 
types  that  cause  false  alarm  detections  in  the  SAR  imagery.  Models  of  the  spatial  distribution  of 
these  plants  were  then  used  to  discriminate  vegetation-caused  SAR  detections  from  those  caused 
by  metallic  objects.  The  HSI  final  report  provides  more  information  on  vegetation  modeling 
methodologies  (Foley  et  ah,  2007a). 

3,5,4  Data  Fusion 

Multiple-sensor  data  fusion  is  a  key  element  in  the  successful  assessment  of  surface  metal 
patterns  with  SAR  technology.  Data  fusion  was  performed  at  several  levels  of  abstraction  within 
the  WAA  analysis  for  this  demonstration,  including  1)  image-level  fusion  of  HSI,  LiDAR,  and 
orthophotography  data  to  extract  vegetation  features  (as  described  above);  2)  image-level  fusion 
of  optical  and  SAR  imagery  from  multiple  look  directions  to  prescreen  and  full,  image-based 
classifiers  that  predict  the  image  positions  of  metallic  signal  generators;  and  3)  abstraction  of 
spatial  models  that  overlay  classified  object  detections  from  the  various  datasets  and  image- 
based  classifiers  to  delineate  the  geospatial  patterns  of  metal  object  detections  weighted  by 
classification  certainty  and  filtered  by  false  alarm  object  detections.  The  HSI  and  SAR  final 
reports  provide  more  information  on  data  fusion  methodologies  (Foley  et  ah,  2007a  and  2007b). 
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PERFORMANCE  ASSESSMENT 


4.1  PERFORMANCE  DATA 

Primary  performance  criteria  included  the  qualitative  assessment  of  the  ease  of  technology  usage 
and  the  quantitative  assessment  of  positional  accuracy  and  resolution  of  both  datasets.  Secondary 
performance  objectives  considered  the  contribution  of  HSI  to  the  WAA  efforts,  including  SAR 
FAR  mitigation  through  vegetation  modeling,  the  detection  of  surface  metal,  and  the  probability 
of  detection  (Pd)  of  target  features  and  other  large  munitions-related  features.  Performance  data 
are  provided  in  Tables  4  and  5. 

4.1.1  Spatial  Accuracy 

The  spatial  accuracy  assessment  SAR  imagery  was  performed  by  comparing  image  target 
locations  with  the  corresponding  top  hat  fiducial  locations  defined  by  precision  GPS  coordinates 
determined  when  the  fiducials  were  placed.  Table  3  provides  summary  statistics  of  these 
accuracies.  By  registering  eight  cardinal  pass-direction  images  per  tile  to  the  orthophotography 
and  to  each  other,  the  horizontal  error  in  the  SAR  data  was  improved  to  less  than  1  pixel  (0.25- 
m).  This  level  of  spatial  accuracy  is  able  to  support  SAR  target  discrimination  using  LiDAR, 
orthophotography,  and  HSI  auxiliary  sensor  data. 

Table  3,  Summary  of  SAR  Radial  Error  and  Post-Registration  Residual  Errors, 


Seven  SAR  Look  Directions 

Average 

(m) 

Standard 
Deviation  (m) 

Y  root  mean  square  error  (RMSE) 

0.18 

Y  linear  error  (95%) 

0.36 

_ 0T2 _ 1 

XRMSE 

0.23 

X  linear  error  (95%) 

0.45 

_ 0T8 _ 1 

RMSE  horizontal  radial  error  (68.3%  eonfidenee) 

0.30 

Horizontal  radial  error  (95%  eonfidenee  level) 

0.50 

0.12 

Spatial  accuracy  of  the  GLT-trans formed  HSI  imagery  was  formally  evaluated  in  a  subset  of  the 
21  flight  strips  by  locating  the  positions  of  ground  fiducial  markers  in  the  geocorrected  HSI 
imagery,  then  computing  the  X-  and  Y-offsets  from  the  known  GPS  location  of  each  marker. 
Table  6  shows  the  results  of  the  assessment  on  three  flight  strips.  While  the  GLT-processed 
spatial  error  was  adequate  for  large-feature  detection  and  characterization,  the  radial  RMSE 
should  be  less  than  1-m  in  order  for  the  data  to  be  useful  for  detecting  small  metal  objects  or  for 
fusion  processing  with  SAR,  LiDAR,  and  orthophotography  data.  Therefore,  a  principal 
components  spectral  sharpening  algorithm  was  used  to  increase  spatial  resolution  through  fusion 
with  the  orthophoto  image,  which  successfully  increased  the  HSI  image  resolution  to  0.25-m. 
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Table  4,  SAR  Performance  Data. 


Type  of 

Performance  Objective 

Performance 

Criteria 

Performance  Confirmation 
Method 

Performance 

Quantitative 

Georeference  position  accuracy 

Horizontal  accuracy  relative  to 
established  ground  control 

Horizontal  error  of  geocorrected  imagery: 
0.5-m 

Horizontal  error  of  coregistered  imagery: 
0.25-m 

System  noise  level  quantification 

Additive  noise  (NEoO*):  radar 
equation  analysis 

Multiplieative  Noise  Ratio  (MNR): 
]VtNR=  1/clutter-to-noise  ratio 
(CNR) 

NEaO:  <-50  dB 

MNR:  <-10  to -14  dB 

SAR  response  characteristics  of  emplaced 
simulated  munitions  targets  on  the  ground 
surface  within  the  3.67  acre  simulated 
ordnance  area 

Comparison  of  results  to  GPS 
locations 

•  All  simulated  target  sizes  ranging 
from  approximately  13-in  x  3-in  to 

60-in  X  18-in  detected  (all  SNR  levels) 

•  85  %  detection  rate  with  about  60  false 
alarms  using  the  combined  8-look 

SAR  data 

SAR  response  characteristics  of  in  situ 
items  (metal  and  vegetation)  surveyed 
within  the  in  situ  calibration  area. 

Comparison  of  results  to  site  visit 
data  of  GPS  locations  of  metal  and 
vegetation 

•  21  %  of  surface  targets  detected  (at 

SNR  levels  of  2.0  or  greater) 

•  Rate  varied  from  84%  to  about  5% 
with  a  corresponding  number  of  false 
alarms  from  1,800  to  80  at  SNR 
thresholds  of  5.5  to  1.0 

•  Detection  limit:  near  the  magnitude 
threshold  of  about  100  for  metal 
objects  with  volumes  smaller  than  ~ 

10  liters 

*  NEctO  =  noise  equivalent  backscatter. 


Table  5.  HSI  Performance  Data, 


Type  of 
Performance 
Objective 

Performance 

Criteria 

Expected  Performance 
(Metric) 

Performance 
Confirmation  Method 

Actual  Performance 
Metric  Met? 

Qualitative 

(Primary) 

Ease  of  use  and  effieieney 
of  operations 

Effieieney  and  ease  of  use  meets  design 
speeifieations 

General  observations 

Easy  to  use 

Georeference  position 
accuracy 

HSI:  3-4.5-mRMSE 

Comparison  of  datasets 
with  ortho-imagery  of 
known  aeeuraey 

HSI:  4.9-1 1.8-mRMSE 

Quantitative 

(Primary) 

Spatial  resolution 

2004:  1.5-  and  3-m 

2005:  1.5-m 

Direet  examination  of  the 
delivered  datasets 
following  geo-refereneing 
using  supplied  GET  files 

2004:  1.5-  and  3-m 

2005:  1.5-m 

Speetral  resolution 

126  bands  (0.45  pm  -  2.5  pm) 

Direet  examination  of  the 
delivered  datasets 
following  geo-refereneing 
using  supplied  GET  files 

126  bands 
(0.45  pm -2.5  pm) 

SAR  FAR 

Redueed  FAR 

Reeeiver  operating 
eharaeteristie  (ROC) 
evaluation  of  FAR 

Aehieved 

Quantitative 

(Secondary) 

Metal  deteetion  of  surfaee 
metal  objeets  >  1,500 
grams 

>0.8 

Pereentage  of  metal 
features  extraeted  from 
ground  ealibration  and 
validation  datasets  deteeted 
in  HSI 

Surfaee  metal  deteetion  not 
aehievable  at  this  site 

Deteetion  of  target 
features  and  other  large 
munitions-related  features 

>0.9 

Pereentage  of  total  large- 
feature  munitions  response 
sites  identified  in  other 
ground  and  WAA  sensor 
datasets 

0.46 

Table  6,  Horizontal  Accuracy  of  Selected  GLT-Geocorrected  HSI  Flight  Strips, 


Accuracy  Metric 

FL4  (m) 

FL5  (m) 

FL6  (m) 

YRMSE 

11.71 

3.00 

1.09 

Y  linear  error  (95%) 

22.94 

5.89 

2.14 

XRMSE 

0.99 

7.64 

4.73 

X  linear  error  (95%) 

1.94 

14.96 

9.27 

RMSE  horizontal  radial  error  (68.3%  confidence  level) 

11.75 

8.20 

4.85 

Horizontal  radial  error  (95%  confidence  level) 

15.54 

13.02 

7.12 

4,1.2  SAR  Target  Detection 

The  ability  of  SAR  to  detect  an  object  depends  on  both  the  RCS  of  the  object  and  the  amount  of 
clutter  in  the  resolution  cell.  Optimally  for  target  detection,  the  RCS  is  large  and  the  clutter  is 
small.  RCS  depends  in  large  part  on  the  size  and  shape  of  the  object.  It  drops  rapidly  when  the 
object  is  smaller  than  the  wavelength  of  the  radar  signal,  and  very  rapidly  when  the  object  is  less 
than  about  1/10  of  the  wavelength;  the  SkySAR  wavelengths  range  from  0.68-m  to  1.3-m. 
Clutter,  on  the  other  hand,  is  an  integrated  effect;  its  signature  represents  the  total  energy  from  all 
the  other  items  in  a  resolution  cell  plus  additional  radar  error.  Therefore,  based  on  the  SkySAR 
image  noise  and  resolution  characteristics,  an  object  should  be  detected  from  any  single  SkySAR 
image  if  its  RCS  peak  amplitude  is  between  10-14  dB  above  the  surrounding  clutter.  For  this 
demonstration,  this  suggests  that  surface  metal  objects  about  30-cm  across  should  be  detectable 
in  the  data,  and  single  metal  objects  on  the  surface  smaller  than  about  15 -cm  in  diameter  will 
likely  disappear  in  the  clutter.  This  conclusion  was  supported  by  data  results  from  the  two 
calibration  sites  evaluated  for  this  demonstration. 

The  results  of  the  SAR  detection  analysis  executed  from  data  collected  over  the  Simulated 
Ordnance  Calibration  Area  are  presented  in  Table  7.  A  total  of  28  simulated  ordnance  items  were 
emplaced,  six  of  which  were  placed  under  cacti  to  help  assess  whether  such  arrangements  could 
be  detected  by  SAR.  The  results  showed  that  these  items  could  not  be  used  for  the  SAR 
processing  and  image  formation  methods  used  for  this  demonstration.  Using  the  prescreener,  a 
100%  detection  rate  of  the  22  detectable  simulated  ordnance  items  was  achieved  while  passing 
approximately  280  false  alarms,  with  results  for  a  100%  detection  threshold  illustrated  in  Figure 
8  as  a  POI  overlay  on  an  orthophoto  image. 

Table  7,  Simulated  Ordnance  Calibration  Area  Items  Detected  by  SAR, 


#of 

Ordnance 

Items 

Length  (m) 

Width  (m) 

Volume  (L) 

1 

1.52 

.46 

197.30 

1 

.91 

.46 

118.33 

1 

.91 

.46 

118.33 

3 

.71 

.22 

81.00 

1 

.74 

.22 

54.01 

2 

.74 

.22 

27.01 

1 

.71 

.22 

27.01 

1 

.46 

.15 

14.79 

1 

.33 

.08 

12.09 
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Table  7,  Simulated  Ordnance  Calibration  Area  Items  Detected  by  SAR,  (continued) 


#of 

Ordnance 

Items 

Length  (m) 

Width  (m) 

Volume  (L) 

1 

1.52 

.08 

6.16 

1 

1.52 

.06 

5.87 

1 

1.24 

.06 

3.11 

1 

1.22 

.06 

2.93 

1 

1.22 

.06 

2.93 

1 

1.09 

.06 

2.93 

1 

1.22 

.08 

2.93 

1 

1.02 

.06 

2.77 

1 

.71 

.09 

2.35 

1 

.71 

.09 

2.35 

X  IQ®  Prescreener  results  on  150m  x  150m  area  around  simulated  UXOs 

-4.1698 


-4.1698 


-4.1697 


-4.1697 


-4.1697 


-4.1697 


-4.1697 


-4.1696 

6.1588  6.159  6.1592  6.1594  6.1596  6.1598  6.16  6.1602 

X  10® 

Figure  8,  Prescreener  Output  (POIs)  in  tbe  Vicinity  of  tbe  Simulated  UXO,  at  Probability 
of  Detection  (Pd)  =  100%,  (Simulated  UXO  locations  are  identified  by  yellow  eireles; 
prescreener  POIs  are  displayed  as  blue  points.) 
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Table  8  lists  the  metal  objeets  in  the  in  situ  ealibration  area  that  were  deteeted;  the  remaining  23 
metal  objeets  known  to  be  present  in  the  area  (as  doeumented  in  a  site  visit  to  the  area)  were  not 
of  suffieient  size  or  SNR  level  to  be  deteeted  by  the  SAR  teehnology. 

Table  8,  In  Situ  Metal  Objects  Detected  by  SAR. 


Length 

Width 

Volume 

(m) 

(m) 

(L) 

0.40 

0.20 

24.0 

0.30 

0.30 

18.0 

0.30 

0.20 

18.0 

0.46 

0.28 

16.7 

0.25 

0.25 

15.6 

0.30 

0.30 

13.5 

A  set  of  ROC  eurves  for  the  two  ealibration  areas  was  developed  to  investigate  target  deteetion 
and  FARs  in  the  SAR  imagery  versus  known  metal  ealibration  objeets  (Foley  et  ah,  2007b).  To 
exeeute  this  analysis,  the  eoregistered  8-look  SAR  imagery  was  exploited  and  a  single  SAR 
image  eombining  all  8-looks  developed.  For  both  the  simulated  ordnanee  and  in  situ  ealibration 
areas,  the  8-look  SAR  image  signilieantly  outperformed  any  individual  SAR  look  image.  In  the 
simulated  ordnanee  area,  an  85%  deteetion  rate  was  aehieved  with  only  60  false  alarms  (Figure 
9).  By  eomparison,  a  single-look  SAR  produeed  about  800  false  alarms  to  aehieve  an  85% 
deteetion  rate.  Beeause  of  the  low  number  of  deteetable  targets  in  the  in  situ  ealibration  area,  the 
ROC  eurves  were  less  useful  in  deseribing  the  results.  The  deteetion  rate  for  the  six  items 
deteetable  by  the  SAR  teehnology  varied  from  84%  to  about  5%  with  a  eorresponding  number  of 
false  alarms  from  1,800  to  80.  Clearly,  at  low  thresholds,  the  deteetions  were  assoeiated  with 
noise  in  the  data. 

4.1,3  False  Alarm  Mitigation 

Vegetation  modeling  results  were  used  to  develop  diserimination  masks.  The  utility  of  the 
diserimination  masks  were  evaluated  relative  to  the  developed  SAR  target  datasets  to  derive 
suitable  index  value  thresholds  for  deteeting  and  diseriminating  vegetation  and  slope-related 
SAR  targets.  Eaeh  SAR  target  was  reviewed  with  respeet  to  eaeh  diserimination  mask.  The  ROC 
eurves  for  the  simulated  ordnanee  ealibration  area  showed  that  more  than  an  order  of  magnitude 
reduetion  in  rate  was  aehieved  when  vegetation  masks  were  utilized  (Foley  et  ah,  2007a  and 
2007b). 

4.2  DATA  FUSION 

In  addition  to  the  vegetation  modeling  results,  two  additional  data  fusion  results  were  aehieved 
by  exploiting  the  registered  SAR  images,  orthophoto  data,  HSI  data,  ground  truth  for  the 
simulated  metal  ordinates,  and  height  of  vegetation  determined  from  the  LiDAR  data.  Using  the 
GGobi  software  system,  2-D,  3-D,  and  multidimensional  seatter  plots  were  viewed  in  different 
eombinations.  For  the  3-D  and  multidimensional  seatter  plots,  the  data  were  rotated  about 
different  axes  to  determine  the  optimal  separation  of  metal  points  from  the  vegetation  and  dirt 
elutter.  The  most  sueeessful  eombination  for  separating  metal  objeets  from  the  baekground 
elutter  was  adding  the  orthophoto  RGB  data  and  HSI  data  to  the  SAR  data. 
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Figure  9,  ROC  Curve  for  the  Combined  8-Look  SAR  Data  in  the 
Simulated  Ordnance  Calibration  Area, 


4,3  HSI  DETECTION  OF  TARGET  FEATURES  AND  OTHER  LARGE  FEATURES 

The  strategy  of  using  HSI  imagery  to  deteet  and  map  large  munitions-related  features  sueh  as 
target  eircles  and  disturbed  ground  areas  proved  to  be  a  useful  adjunct  to  the  WAA  analysis 
using  LiDAR  and  orthophotography.  The  HSI  data  contributed  significantly  to  the  detection  and 
confirmation  for  two  of  the  five  ship  targets  observed  in  the  vicinity  of  BT4  because  they  were 
topographically  obscured  in  the  LiDAR  data  and  very  difficult  to  detect  in  the  orthophotography 
data  (Figure  10). 

However,  because  of  reduced  spatial  resolution  and  accuracy,  HSI  data  alone  would  not  be  a 
successful  substitute  for  either  LiDAR  or  orthophotography  data  sets.  The  summary  in  Table  9 
shows  the  results  using  HSI  data;  the  detection  rate  for  ship  targets  is  relatively  high  and  for 
smaller  disturbance  features  relatively  low. 
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Figure  10,  The  Ship  Target  Obscured  in  the  LIDAR  Microshaded  Relief  hy 
Cratering  (left)  Is  Clearly  Visible  in  the  HSI  Data  (right). 


Table  9,  Large  Feature  Detection  Comparison, 


Feature  Type 

Observed  in  HSI 

Total  Features 
Observed 

Percent 

Detection 

Target  circles 

3 

4 

0.75 

Target  crosses 

0 

4 

0.00 

Ship  targets 

4 

5 

0.80 

Other  significant  features 

3 

10 

0.33 

Total  large  features 

10 

23 

.44 

4.4  HSI  METAL  DETECTION 

The  calibration  studies  described  did  not  succeed  in  using  HSI  data  to  directly  detect  sub-pixel- 
sized  metal  features  on  the  landscape  surface.  Consequently,  the  surface  metal  detection 
objective  of  the  demonstration  was  concluded  with  no  further  site-wide  analysis. 

4.5  DATA  ASSESSMENT 

In  terms  of  detection,  the  developed  SAR  technology  showed  robust  detection  of  targets 
emplaced  on  the  ground  surface  in  the  Simulated  Ordnance  Calibration  Area; 

•  Over  80%  of  the  emplaced  targets  were  detected  from  a  single  look-direction; 
when  all  look  directions  were  exploited,  100%  of  all  targets  were  detected. 

•  Using  a  SAR  image  that  combines  ail  look  directions,  100%  detection  was 
achieved  with  120  false  alarms,  which  was  an  order  of  magnitude  reduced 
compared  to  single  look  results.  False  alarms  were  reduced  to  about  60  while 
maintaining  an  85%  detection  rate. 

•  The  FAR  was  reduced  through  data  fusion  methods  that  exploited  vegetation 
modeling  masks. 
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However,  the  results  were  not  as  eneouraging  for  the  smaller-sized  items  generally  present  in  the 
In-Situ  Calibration  Area.  The  SAR  teehnology  eapability  demonstrated  a  deteetion  limit  near  the 
magnitude  threshold  of  about  100  for  metal  objeets  with  volumes  smaller  than  about  10  liters. 

Overall,  the  results  indieate  that  the  detection  of  concentrated  munitions  debris  is  possible  in 
areas  of  low  background  noise  (vegetation,  water,  or  rock  facets)  and  with  fragments  or 
munitions  near  the  size  threshold  for  detection.  At  an  appropriate  site  (such  as  a  dry  lake  range) 
that  had  not  been  surface  cleared,  the  SAR  technology  would  likely  clearly  delineate  “clean” 
areas  unlikely  to  have  MEC  contamination. 

For  other  sites,  “density  contrast”  is  a  way  to  characterize  the  issue  of  detecting  small-item 
density  clusters  against  a  noisy  background.  In  this  demonstration,  false  alarm  filtering  methods 
were  used  to  leverage  secondary  sensors  (used  successfully  at  less  challenging  sites)  to  detect 
and  eliminate  false  alarms  that  were  of  significantly  greater  signal  magnitude  than  munitions 
debris  items,  but  were  hampered  by  the  sheer  numbers  of  false  alarm  generators  (primarily 
cactus)  at  this  site.  In  addition,  if  the  cactus  and  other  highly  SAR-responsive  plants  were  not 
detected  and  eliminated  and  were  instead  part  of  the  background  noise,  then  the  smaller  metallic 
items  could  not  be  detected  individually.  It  is  unknown  and  untested  what  density  of  small 
munitions  fragments  would  be  required  to  provide  sufficient  contrast  for  detection  against  a 
vegetation  background  like  that  encountered  at  the  demonstration  site,  but  it  may  be  likely  that 
such  concentrations  do  not  commonly  occur  in  actual  munitions  sites. 

HSI  was  found  to  be  an  important  dataset  for  data  fusion,  primarily  the  creation  of  vegetation 
models  for  FAR  mitigation.  In  addition,  using  HSI  imagery  to  detect  and  map  large  munitions- 
related  features  such  as  target  circles  and  disturbed  ground  areas  proved  to  be  a  useful  adjunct  to 
the  FiDAR  and  large-scale  orthophotography  datasets.  Although  HSI  data  alone  would  not  be  a 
successful  substitute  for  either  FiDAR  or  orthophotography  datasets,  it  can  play  a  part  in  WAA 
analysis  using  high-airborne  remote  sensing  techniques.  In  this  demonstration,  HSI  was  not  able 
to  be  used  for  surface  metal  detection  due  to  the  lack  of  spectral  contrast  between  materials  of 
interest  and  natural  landscape  materials. 

4,6  TECHNOLOGY  COMPARISON 

The  ability  of  SAR  and  HSI  technologies,  both  individually  and  using  HSI  for  false-alarm- 
reduction,  was  evaluated  for  the  detection  of  metal  of  munitions,  and  munitions  debris  was 
evaluated  in  this  demonstration.  The  most  appropriate  technology  for  comparison  is  helicopter 
magnetometry  technology. 

Because  SAR  and  HSI  data  can  be  collected  from  fixed-wing  aircraft  at  normal  flight  speed  and 
altitude,  these  technologies  can  be  used  to  survey  thousands  of  acres  per  day.  This  compares  to 
helicopter  magnetometry  technology,  which  can  be  used  to  survey  on  average  400-500  acres  per 
day.  In  addition,  helicopter  technology  is  limited  to  areas  where  vegetation,  if  present,  is  of 
limited  height  (approximately  less  than  2-m)  and  to  areas  without  steep  topography.  Therefore, 
these  technologies  may  be  used  in  areas  where  low-altitude  helicopter  surveys  for  the  detection 
of  munitions  would  not  be  feasible.  Additional  testing  and  analysis  would  be  required  to  confirm 
whether  they  can  be  successfully  applied  at  vegetated  sites  and  must  take  into  account  the  fact 
that  SAR’s  long  wavelengths  limit  the  targets  that  can  be  detected.  Lowering  the  threshold  to 
detect  munitions  and  munitions  debris  would  result  in  a  commensurate  increase  in  the  FAR.  In 
addition,  the  data  processing,  analysis,  and  fusion  techniques  used  for  this  demonstration  are  not 
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as  yet  as  standard  as  the  proeessing  and  analysis  teehniques  for  helieopter  magnetometry  surveys 
and  therefore  are  more  time-eonsuming  and  expensive. 
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5.0  COST  ASSESSMENT 


5.1  COST  REPORTING 

Cost  information  associated  with  the  demonstration  of  all  airborne  teehnology  and  assoeiated 
aetivities  was  traeked  and  doeumented  before,  during,  and  after  the  demonstrations  to  provide  a 
basis  for  determination  of  the  operational  costs  associated  with  this  teehnology.  For  these 
demonstrations.  Tables  10  and  11  eontain  the  cost  elements  that  were  tracked  and  doeumented 
for  this  demonstration.  These  eosts  inelude  both  operational  and  eapital  eosts  assoeiated  with  the 
demonstration  design  and  planning;  salary  and  travel  eosts  for  support  staff;  equipment  costs 
associated  with  aireraft  and  sensors;  support  personnel;  and  eosts  assoeiated  with  the  proeessing, 
analysis,  and  interpretation  of  the  results  generated  by  this  demonstration. 

5.2  COST  ANALYSIS 

The  eost  of  an  airborne  survey  depends  on  many  faetors,  ineluding: 

•  Aireraft  eosts.  The  rental  rate  for  aireraft  eosts  will  be  influenced  in  part  by  the 

type  of  aireraft  utilized  and  the  eost  of  fuel  (if  ineluded  in  the  eost  of  the  aireraft 
time).  In  addition,  standby  time,  if  needed,  will  inerease  the  survey  eosts. 

•  Length  and  number  of  flight  lines  required  to  survey  the  demonstration  area. 

•  Aeeuraey  requirements,  whieh  influenee  the  speed  and  altitude  of  the  survey 

flights  and  the  amount  of  data  proeessing  required. 

•  Loeation  of  the  site,  whieh  ean  influence  the  cost  of  mobilization  and  logistics. 

•  Amount  of  analysis  required  to  suffieiently  proeess  and  analyze  the  data. 

Costs  assoeiated  with  the  ground  validation  surveys  to  eolleet  post-survey  data  were  not 
eonsidered  in  the  eost  analysis,  as  the  validation  was  eonducted  as  part  of  the  WAA  pilot 
program.  Also,  this  demonstration  was  one  of  several  eoneurrent  demonstrations  condueted  by 
Sky  Researeh  for  the  WAA  pilot  program.  Some  of  the  eosts  assoeiated  with  management  were 
shared  among  demonstrations,  lowering  some  of  the  eosts  ineurred  for  this  demonstration. 

Aireraft  eosts  are  a  major  eost  faetor  for  any  airborne  survey.  Signifieant  variables  and  faetors 
assoeiated  with  the  mobilization,  data  aequisition,  and  demobilization  eosts  inelude  the  eost  of 
aireraft  time  and  standby  time.  The  eost  of  aireraft  ean  vary  depending  on  the  type  of  aireraft  and 
operating  eosts.  Standby  time  ean  also  influenee  the  eost  of  a  survey  and  is  typieally  assessed  at 
the  eost  of  one  day  of  data  eollection,  ineluding  aireraft  eosts,  labor,  and  travel. 
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Table  10,  Pueblo  Precision  Bombing  Range  SAR  Cost  Tracking,^ 


Cost  Category 

Subcategory 

Details 

Costs  ($) 

Start-Up  Costs 

Predeployment  and  planning 
includes— planning, 
contracting,  presurvey  site 
visits 

Labor — contracting 

Labor— planning 

Labor— site  visit 

Travel 

13,226 

Materials  and  supplies 

608 

Subtotal 

Predeployment  Planning 

Mobilization— personnel 
mobilization,  equipment 
mobilization,  and 
transportation 

Aircraft  time  (2  hours) 

5,122 

Aircraft  modifications 

4,848 

Labor 

Travel 

9,173 

Postage 

Subtotal  Mobilization 

37,733 

Total  Start-Up  Costs 

272,907 

Operating  Costs 

High  airborne  survey— data 
acquisition  and  associated 
tasks,  including  aircraft 
operation  time 

Aircraft  time  (36.8  hours) 

94,225 

Equipment 

49,346 

Labor 

57,392 

Travel 

17,653 

Total  Operating  Costs 

218,616 

Demobilization 

Demobilization— personnel 
demobilization,  equipment 
demobilization 

Aircraft  time  (2  hours) 

5,122 

Labor 

9,279 

Travel 

18,827 

Total  Demobilization  Costs 

33,228 

Data  Processing 

Data  processing 

Labor 

Data  Analysis 

Data  analysis 

Labor 

Supplies  and  materials 

Data  fusion 

Labor 

115,301 

Total  Data  Processing  and  Analysis  Costs 

346,013 

Reporting  and 
Management 

Management  and  reporting 

Labor— reporting 

Labor— management 

40,204 

Travel 

3,570 

Materials  and  supplies 

220 

Total  Reporting  and  Management  Costs 

84,053 

Total  Costs 

954,817 

Acre  Snrveyed 

2,225 

Unit  Cost/Acres  Snrveyed* 

429/acre 

*  Note:  The  unit  cost  includes  the  cost  for  survey  flights  of  the  2,225  acres  and  the  image  formation  for  the  survey  area.  However,  only  a  subset 
of  the  survey  area  data  was  analyzed  (6  acres)  to  evaluate  and  demonstrate  SAR  methodology. 


1 


All  costs  reported  for  the  demonstration  include  overhead  and  organization  burden  and  fees. 
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Table  11,  Pueblo  Precision  Bombing  Range  HSI  Cost  Tracking,^ 


Cost  Category 

Sub  Category 

Details 

Costs  ($) 

Start-Up  Costs 

Predeployment  and 
planning— includes 
planning,  contracting, 
presurvey  site  visit 

Labor 

27,053 

Site  visit  travel 

3,428 

Subtotal 

Predeployment  Planning 

30,481 

Mobilization— personnel 
mobilization  and  equipment 
mobilization 

Subconhactor  (mobilization  of 
equipment,  aircraft  and 
personnel) 

11,103 

Labor 

4,137 

Subtotal  Mobilization 

15,240 

Total  Start-Up  Costs 

45,721 

Operating  Costs 

High  airborne  survey— data 
acquisition  and  associated 
tasks,  including  aircraft 
operation  time 

Subconhactor  (equipment, 
aircraft  and  personnel) 

37,010 

Labor 

7,985 

Materials  and  supplies 

Total  Operating  Costs 

45,875 

Demobilization 

Demobilization— personnel 
demobilization,  equipment 
demobilization 

Subconhactor  (mobilization 
of  equipment,  aircraft  and 
personnel) 

11,103 

Labor 

4,137 

Total  Demobilization  Costs 

15,240 

Data  Processing  and 
Analysis 

Data  processing  and 
analysis,  including 
vegetation  modeling 

Labor 

96,115 

Total  Data  Processing  and  Analysis  Costs 

83,558 

Management  and 
Reporting 

Management  and  reporting 

Labor 

52,007 

Travel 

14,107 

Total  Management  and  Reporting  Costs 

66,114 

Total  Costs 

269,065 

Acres  Characterized 

13,334 

Unit  Cost/Acre  Characterized 

20.18/acre 

*  Note:  3,044  acres  of  the  2005  data  collection  overlapped  the  2004  data  collection  area.  The  13,334  acres  is  the  total  number  of  acres 
characterized. 


^  All  costs  reported  for  the  demonstration  include  overhead  and  organization  burden  and  fees. 
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Project  planning  was  a  significant  cost  associated  with  the  SAR  demonstration  because  of  the 
need  to  reengineer  the  SAR  sensor  prior  to  deployment.  In  addition,  due  to  the  unique  nature  of 
the  demonstration,  more  planning  meetings  with  ESTCP  personnel  were  conducted  than  would 
be  typically  required  of  a  production-level  survey. 

Mobilization  and  demobilization  costs  are  most  significantly  a  function  of  the  distance  from  the 
home  base  for  the  aircraft.  In  addition  to  the  cost  of  mobilizing  and  demobilizing  the  aircraft,  the 
cost  of  mobilizing  equipment  (sensors  and  GPS  equipment)  can  add  significantly  to  costs.  For 
these  demonstrations,  the  daily  rates  for  the  SAR  sensor  and  GPS  equipment  were  not  charged 
for  mobilization  and  demobilization  because  each  required  less  than  one  day.  For  a  mobilization 
taking  a  full  day  or  longer,  the  daily  rate  for  the  SAR  sensor  and  associated  equipment  would 
have  been  assessed,  increasing  the  mobilization  and  demobilization  costs.  Therefore,  for  a  site 
requiring  a  longer  mobilization  distance,  the  mobilization  and  demobilization  can  take  up  a 
correspondingly  larger  amount  of  the  budget,  especially  when  considered  as  a  percentage  of  the 
cost  for  surveys  of  relatively  small  areas. 

For  the  costs  associated  with  the  data  acquisition  flights,  the  predominant  costs  are  the  cost  of  the 
aircraft  and  equipment.  The  major  driver  for  these  costs  is  the  survey  size.  Fixed-wing  aircraft 
can  cover  up  to  approximately  15,000  acres  per  day  on  average.  Sites  less  than  this  in  size  will 
therefore  be  more  expensive  on  a  per  acre  basis.  The  remainder  of  costs  associated  with  data 
acquisition  includes  labor,  travel,  and  materials/supplies. 

Data  processing  and  analysis  costs  are  typically  linear  with  project  size.  For  these 
demonstrations,  the  data  processing  and  analysis  techniques  were  still  under  development, 
causing  the  processing  and  analysis  costs  to  be  much  higher  than  would  typically  be  encountered 
on  a  production-level  survey.  In  addition,  environmental  conditions  such  as  the  vegetation 
encountered  increased  the  amount  of  time  needed  for  vegetation  modeling  and  data  fusion.  Sites 
without  vegetation  would  require  less  analysis.  Processing  and  analysis  costs  have  been 
decreasing  with  experience  at  multiple  sites,  automation  of  processing  and  analysis  routines  and 
increased  computing  power  resulting  in  faster  processing. 

Project  management  and  reporting  were  a  somewhat  significant  cost  for  these  demonstrations,  as 
the  projects  were  conducted  under  the  WAA  pilot  program  and  required  more  meetings  and 
reporting  than  would  generally  be  expected  for  a  production  level  survey. 

5.3  TYPICAL  AIRBORNE  SURVEY  COSTS 

Mobilization  distance,  site  size,  site  conditions,  and  project  objectives  can  influence  the  costs  of 
data  collection  and  analysis.  In  addition,  this  demonstration  utilized  a  fixed-wing  aircraft; 
different  kinds  of  fixed-wing  aircraft  will  have  somewhat  different  hourly  rates  than  the  rates 
used  in  this  demonstration. 

Typical  airborne  survey  costs  for  SAR  would  be  less  than  encountered  on  this  demonstration 
because  of  the  advances  in  technology  since  this  demonstration  was  conducted,  most  notably  in 
the  area  of  data  processing  and  analysis  costs.  For  SAR  data  collection,  data  processing,  analysis 
(SAR  data  only),  and  reporting  of  a  survey  site  of  75,000  acres  (three  days  of  data  collection), 
the  costs  would  be  approximately  on  the  order  of  $570,000  (or  $7. 70/acre).  Costs  for  performing 
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data  fusion  with  additional  datasets  are  not  ineluded  in  this  estimate  and  would  inerease  the  cost 
of  a  SAR  survey  if  advanced  analysis  was  desired. 

5.4  COST  CONCLUSIONS 

Due  to  the  unique  nature  of  the  demonstration  and  the  need  to  reengineer  the  SAR  sensor, 
develop  and  investigate  data  processing  and  analysis  techniques,  and  investigate  data  fusion 
techniques,  the  cost  per  acre  for  SAR  was  much  higher  than  for  other  high-airborne  survey 
technologies.  These  costs,  however,  would  be  expected  to  be  much  lower  with  a  more  mature 
system  in  a  production-level  survey. 

A  number  of  factors  should  be  considered  for  WAA  technology  selection,  including  the 
acquisition  of  SAR  and  HSI  data,  when  evaluating  the  appropriateness  of  airborne  technologies 
and  potential  for  cost  savings.  For  effective  use  of  resources,  site  characteristics  should  be 
evaluated  to  determine  the  use  of  appropriate  sensors.  Site  characteristics  of  concern  for  the 
application  of  SAR  and  HSI  technologies  include  site  size,  terrain,  vegetation,  and  contamination 
characteristics.  In  sites  with  vegetation,  if  SAR  is  to  be  used  for  metal  detection,  HSI-derived 
vegetation  models  will  most  likely  be  necessary  for  false-alarm  reduction,  increasing  costs. 
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6.0  IMPLEMENTATION  ISSUES 


6.1  COST  OBSERVATIONS 

Because  of  the  need  to  reengineer  the  SAR  system  and  develop  data  processing  and  analysis 
techniques  to  analyze  the  SAR  data,  this  demonstration  was  expensive  in  comparison  to  better 
understood  and  more  widely  used  WAA  technologies.  However,  since  the  time  of  this 
demonstration.  Sky  Research  has  further  refined  the  SkySAR  system  to  expand  the  sensor 
capabilities  and  increase  efficiency  in  data  processing  and  analysis.  Therefore,  the  costs  to 
deploy  the  SAR  system  would  be  less  than  the  costs  experienced  for  this  demonstration. 

6.2  PERFORMANCE  OBSERVATIONS 

Pueblo  presented  a  challenging  site  for  the  demonstration  of  these  technologies  due  to  the 
vegetation  issues  (SAR)  and  chemical  composition  of  the  soil  (HSI).  In  addition,  as 
demonstrated,  SAR  technology  can  detect  metal  but  is  limited  by  minimum  size  detections. 
Therefore,  it  is  important  to  review  and  assess  the  site  characteristics  and  potential  munitions 
contamination  and  the  potential  impact  conditions  such  as  these  might  have  on  the  results  before 
incorporating  the  use  of  these  sensors  in  a  multisensor  WAA  analysis. 

6.3  SCALE-UP 

As  with  all  airborne  surveys,  the  use  of  either  technology  on  very  small  sites  may  not  be  cost 
effective  because  of  the  costs  associated  with  mobilization,  demobilization,  and  minimum 
operation  times  of  aircraft,  therefore,  the  larger  the  site,  the  more  cost-effective  their  use  on  a  per 
acre  basis. 

6.4  OTHER  SIGNIFICANT  OBSERVATIONS 

The  following  are  observations  from  this  demonstration  and  recommendations  for  continued 
work  with  these  data: 

•  Coregistration.  The  coregistration  was  based  on  multiple  link-points  associated 
with  each  dataset.  While  adequate  for  this  analysis,  the  spatial  coregistration 
process  was  labor-intensive  and  can  be  improved  through  automated  procedures. 
As  use  of  these  data  has  shown  to  be  essential  to  reducing  the  number  of  false 
alarms,  refinement  of  this  process  is  recommended. 

•  Vegetation  model.  The  results  related  to  FARs  demonstrated  the  value  of  the 
derived  site  vegetation  model  based  on  the  fusion  of  orthophotography,  LiDAR, 
and  HSI  data.  The  exploitation  of  these  data  can  be  improved  through  a  more 
comprehensive  analysis,  including  additional  vegetation  ground  truth  data. 

•  Site  calibration  information.  The  Simulated  Ordnance  Calibration  Area  proved  to 
be  very  useful  for  the  definition  of  SAR  detection  capabilities.  However,  this  area 
was  limited  in  terms  of  size,  range  of  site  conditions,  and  number  and  type  of 
emplaced  simulated  munitions.  To  supplement  the  simulated  ordnance  data, 
additional  information  was  gathered  from  an  In  Situ  Calibration  Area. 
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Unfortunately,  this  randomly  selected  area  contained  few  metal  targets  within  the 
operating  wavelengths  of  the  SAR  (68-  to  130-cm).  It  is  recommended  that 
additional  ground  truth  information  related  to  surface  metal  distribution  be 
collected  and  included  in  the  analysis.  In  the  future,  a  significantly  more 
comprehensive  simulated  ordnance  area  should  be  utilized. 

•  Operating  parameter  optimization.  The  ROC  curves  developed  during  this  project 
represented  the  synthesis  of  numerous  data  collection,  processing,  analysis, 
detection,  fusion,  and  discrimination  steps.  The  optimization  of  this  process  is 
recommended  to  maximize  the  ROC  curve  performance  of  the  technology. 

6.5  LESSONS  LEARNED 

Several  lessons  have  been  learned  through  the  execution  of  this  project  regarding  the  use  of  SAR 
as  a  WAA  tool  applied  to  former  military  facilities  contaminated  with  MEC.  Significant  insights 
were  gained  on  the  following  topics: 

•  The  importance  of  multiple-look  SAR  spatial  registration,  their  coregistration 
with  companion  datasets,  and  the  procedures  required  to  achieve  decimeter-level 
accuracy 

•  The  exploitation  of  multiple-look  SAR  data  to  establish  improved  detection  rates 

•  The  effective  and  coordinated  fusion  of  SAR  data  in  conjunction  with  HSI, 
LiDAR,  and  orthophotography  datasets  for  the  mitigation  of  false  alarms 

•  The  feedback  of  ground  verification  information  to  establish  optimal  processing 
parameters  used  in  establishing  effective  vegetation  models,  SAR  filtering 
parameters,  and  multilook  SAR  SNR  thresholds. 

6.6  END-USER  ISSUES 

Implementing  WAA  for  production  level  surveys  should  include  end  users  in  the  project.  For  this 
project,  ESTCP  has  been  utilizing  the  WAA  Advisory  Group  to  understand  and  evaluate 
potential  end-user  issues  and  concerns  that  can  impact  the  widespread  implementation  of  WAA 
technologies.  End  users  can  also  be  provided  on-line  access  to  WAA  data  and  analytical  tools 
through  the  use  of  GIS,  as  demonstrated  in  the  MM-0537  WAA  GIS  demonstration  project. 

6.7  APPROACH  TO  REGULATORY  COMPLIANCE  AND  ACCEPTANCE 

The  ESTCP  Program  Office  established  an  Advisory  Group  to  facilitate  interactions  with  the 
regulatory  community  and  potential  end-users  of  this  technology.  Members  of  the  Advisory 
Group  included  representatives  of  the  USEPA,  state  regulators,  USAGE  officials,  and 
representatives  from  the  services.  ESTCP  staff  worked  with  the  Advisory  Group  to  define  goals 
for  the  WAA  pilot  program  and  develop  Project  Quality  Objectives. 

There  will  be  a  number  of  issues  to  overcome  to  allow  widespread  implementation  of  WAA 
beyond  the  pilot  program.  Most  central  is  the  change  in  mindset  that  will  be  required  if  the  goals 
of  WAA  extend  from  delineating  target  areas  to  collecting  data  that  are  useful  in  making 
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decisions  about  areas  where  there  is  not  indication  of  munitions  use.  Therefore,  the  challenge  for 
adoption  of  a  WAA  approach  with  respect  to  regulatory  acceptance  may  be  the  collection  of 
sufficient  data  and  evaluation  that  the  application  of  these  technologies  to  uncontaminated  land 
and  understanding  the  results.  Similarly,  demonstrating  that  WAA  data  can  be  used  to  provide 
information  on  target  areas  regarding  boundaries,  density,  and  types  of  munitions  to  be  used  for 
prioritization,  cost  estimation,  and  planning  will  require  that  the  error  and  uncertainties  in  these 
parameters  are  well  understood. 
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